{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-colab"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/pathwaycom/pathway/blob/main/examples/notebooks/tutorials/transformers.ipynb\" target=\"_parent\"><img src=\"https://pathway.com/assets/colab-badge.svg\" alt=\"Run In Colab\" class=\"inline\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Installing Pathway with Python 3.10+\n",
        "\n",
        "In the cell below, we install Pathway into a Python 3.10+ Linux runtime.\n",
        "\n",
        "> **If you are running in Google Colab, please run the colab notebook (Ctrl+F9)**, disregarding the 'not authored by Google' warning.\n",
        "> \n",
        "> **The installation and loading time is less than 1 minute**.\n"
      ],
      "metadata": {
        "id": "notebook-instructions"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "%%capture --no-display\n",
        "!pip install --prefer-binary pathway"
      ],
      "metadata": {
        "id": "pip-installation-pathway",
        "cellView": "form"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "id": "1",
      "metadata": {},
      "source": [
        "# A tour of Pathway's transformer classes\n",
        "\n",
        "In this section, we will go through several examples of Pathway transformer classes.\n",
        "This should give you a good overview of how to handle them and how useful they are.\n",
        "\n",
        "We will not go into implementation details, so you are strongly encouraged to read [our introduction](/developers/user-guide/diving-deeper/transformer-introduction/) first.\n",
        "\n",
        "In the following, we are going to see how to use transformer classes to perform [simple operations on a single row](#simple-operations-on-a-single-row), use [transformers as a method](#transformers-as-a-method), and use transformers to [combine several tables at once](#transformer-classes-using-two-different-tables).\n",
        "\n",
        "## Our guinea pig\n",
        "\n",
        "You will experiment on the following table:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "2",
      "metadata": {},
      "outputs": [],
      "source": [
        "from typing import Any\n",
        "\n",
        "import pathway as pw\n",
        "\n",
        "guinea_pig = pw.debug.table_from_markdown(\n",
        "    \"\"\"\n",
        "    | val  | aux\n",
        " 0  | 0    | 10\n",
        " 1  | 1    | 11\n",
        " 2  | 2    | 12\n",
        " 3  | 3    | 13\n",
        " 4  | 4    | 14\n",
        " 5  | 5    | 15\n",
        " 6  | 6    | 16\n",
        " \"\"\"\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "3",
      "metadata": {},
      "source": [
        "## Simple operations on a single row\n",
        "\n",
        "First, you are going to perform simple operations on the table: adding a given number, obtaining the squared value, and performing the sum of two columns.\n",
        "\n",
        "### Adding 10 to each value:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "4",
      "metadata": {},
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "[2024-06-04T06:35:17]:INFO:Preparing Pathway computation\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "            | result\n",
            "^X1MXHYY... | 10\n",
            "^YYY4HAB... | 11\n",
            "^Z3QWT29... | 12\n",
            "^3CZ78B4... | 13\n",
            "^3HN31E1... | 14\n",
            "^3S2X6B2... | 15\n",
            "^A984WV0... | 16\n"
          ]
        }
      ],
      "source": [
        "@pw.transformer\n",
        "class add_ten:\n",
        "    class table(pw.ClassArg):\n",
        "        val = pw.input_attribute()\n",
        "\n",
        "        @pw.output_attribute\n",
        "        def result(self) -> float:\n",
        "            return self.val + 10\n",
        "\n",
        "\n",
        "result = add_ten(guinea_pig).table\n",
        "pw.debug.compute_and_print(result)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "5",
      "metadata": {},
      "source": [
        "As you can see only the column `val` has been taken into account.\n",
        "\n",
        "### Obtaining the squared value of each value:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "6",
      "metadata": {},
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "[2024-06-04T06:35:17]:INFO:Preparing Pathway computation\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "            | result\n",
            "^X1MXHYY... | 0\n",
            "^YYY4HAB... | 1\n",
            "^Z3QWT29... | 4\n",
            "^3CZ78B4... | 9\n",
            "^3HN31E1... | 16\n",
            "^3S2X6B2... | 25\n",
            "^A984WV0... | 36\n"
          ]
        }
      ],
      "source": [
        "@pw.transformer\n",
        "class squared_value:\n",
        "    class table(pw.ClassArg):\n",
        "        val = pw.input_attribute()\n",
        "\n",
        "        @pw.output_attribute\n",
        "        def result(self) -> float:\n",
        "            return self.val * self.val\n",
        "\n",
        "\n",
        "result = squared_value(guinea_pig).table\n",
        "pw.debug.compute_and_print(result)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "7",
      "metadata": {},
      "source": [
        "### Summing two columns"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "8",
      "metadata": {},
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "[2024-06-04T06:35:17]:INFO:Preparing Pathway computation\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "            | result\n",
            "^X1MXHYY... | 10\n",
            "^YYY4HAB... | 12\n",
            "^Z3QWT29... | 14\n",
            "^3CZ78B4... | 16\n",
            "^3HN31E1... | 18\n",
            "^3S2X6B2... | 20\n",
            "^A984WV0... | 22\n"
          ]
        }
      ],
      "source": [
        "@pw.transformer\n",
        "class summing_columns:\n",
        "    class table(pw.ClassArg):\n",
        "        val = pw.input_attribute()\n",
        "        aux = pw.input_attribute()\n",
        "\n",
        "        @pw.output_attribute\n",
        "        def result(self) -> float:\n",
        "            return self.val + self.aux\n",
        "\n",
        "\n",
        "result = summing_columns(guinea_pig).table\n",
        "pw.debug.compute_and_print(result)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "9",
      "metadata": {},
      "source": [
        "Those three results can be obtained by a unique transformer:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "10",
      "metadata": {},
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "[2024-06-04T06:35:17]:INFO:Preparing Pathway computation\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "            | result_add | result_squared | result_sum\n",
            "^X1MXHYY... | 10         | 0              | 10\n",
            "^YYY4HAB... | 11         | 1              | 12\n",
            "^Z3QWT29... | 12         | 4              | 14\n",
            "^3CZ78B4... | 13         | 9              | 16\n",
            "^3HN31E1... | 14         | 16             | 18\n",
            "^3S2X6B2... | 15         | 25             | 20\n",
            "^A984WV0... | 16         | 36             | 22\n"
          ]
        }
      ],
      "source": [
        "@pw.transformer\n",
        "class combined_transformer:\n",
        "    class table(pw.ClassArg):\n",
        "        val = pw.input_attribute()\n",
        "        aux = pw.input_attribute()\n",
        "\n",
        "        @pw.output_attribute\n",
        "        def result_add(self) -> float:\n",
        "            return self.val + 10\n",
        "\n",
        "        @pw.output_attribute\n",
        "        def result_squared(self) -> float:\n",
        "            return self.val * self.val\n",
        "\n",
        "        @pw.output_attribute\n",
        "        def result_sum(self) -> float:\n",
        "            return self.val + self.aux\n",
        "\n",
        "\n",
        "result = combined_transformer(guinea_pig).table\n",
        "pw.debug.compute_and_print(result)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "11",
      "metadata": {},
      "source": [
        "Finally, you can use the new values inside the same transformer to perform more advanced operations:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "12",
      "metadata": {},
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "[2024-06-04T06:35:17]:INFO:Preparing Pathway computation\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "            | result_add | result_double\n",
            "^X1MXHYY... | 10         | 20\n",
            "^YYY4HAB... | 11         | 22\n",
            "^Z3QWT29... | 12         | 24\n",
            "^3CZ78B4... | 13         | 26\n",
            "^3HN31E1... | 14         | 28\n",
            "^3S2X6B2... | 15         | 30\n",
            "^A984WV0... | 16         | 32\n"
          ]
        }
      ],
      "source": [
        "@pw.transformer\n",
        "class reusing_transformer:\n",
        "    class table(pw.ClassArg):\n",
        "        val = pw.input_attribute()\n",
        "\n",
        "        @pw.output_attribute\n",
        "        def result_add(self) -> float:\n",
        "            return self.val + 10\n",
        "\n",
        "        @pw.output_attribute\n",
        "        def result_double(self) -> float:\n",
        "            return self.result_add + self.result_add\n",
        "\n",
        "\n",
        "result = reusing_transformer(guinea_pig).table\n",
        "pw.debug.compute_and_print(result)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "13",
      "metadata": {},
      "source": [
        "## Transformers as a method\n",
        "\n",
        "You are not bound to static computation as transformers provide a way to obtain methods as new values.\n",
        "This is done using the `method` keyword:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "14",
      "metadata": {},
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "[2024-06-04T06:35:18]:INFO:Preparing Pathway computation\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "            | res\n",
            "^X1MXHYY... | 10\n",
            "^YYY4HAB... | 11\n",
            "^Z3QWT29... | 12\n",
            "^3CZ78B4... | 13\n",
            "^3HN31E1... | 14\n",
            "^3S2X6B2... | 15\n",
            "^A984WV0... | 16\n"
          ]
        }
      ],
      "source": [
        "@pw.transformer\n",
        "class method_transformer:\n",
        "    class table(pw.ClassArg):\n",
        "        val: float = pw.input_attribute()\n",
        "\n",
        "        @pw.method\n",
        "        def method_result(self, arg) -> float:\n",
        "            return self.val + arg\n",
        "\n",
        "\n",
        "method_table = method_transformer(guinea_pig).table\n",
        "result = method_table.select(res=method_table.method_result(10))\n",
        "pw.debug.compute_and_print(result)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "15",
      "metadata": {
        "lines_to_next_cell": 2
      },
      "source": [
        "## Transformer Classes using two different tables\n",
        "\n",
        "Now you might want to do something more complicated which requires two different tables.\n",
        "\n",
        "You have a table `matchings` which contains pairs of values `a` and `b` and a table `profiles` which contains the profile of each value of the pairs.\n",
        "You want to compute, for each pair, the sum of the profiles of the values of the pair.\n",
        "\n",
        "First, you need the tables:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "16",
      "metadata": {},
      "outputs": [],
      "source": [
        "profiles = pw.debug.table_from_markdown(\n",
        "    \"\"\"\n",
        "    | profile\n",
        " 0  | 1\n",
        " 1  | 10\n",
        " 2  | 100\n",
        " 3  | 1000\n",
        " \"\"\"\n",
        ")\n",
        "\n",
        "matchings = pw.debug.table_from_markdown(\n",
        "    \"\"\"\n",
        "    | a  | b\n",
        " 0  | 0  | 2\n",
        " 1  | 1  | 3\n",
        " \"\"\"\n",
        ")\n",
        "matchings = matchings.select(\n",
        "    a=profiles.pointer_from(matchings.a), b=profiles.pointer_from(matchings.b)\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "17",
      "metadata": {},
      "source": [
        "Now, you can do a transformer which takes the two tables as parameters.\n",
        "To access a given table inside the transformer, use the notation `self.transformer.my_table`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "18",
      "metadata": {},
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "[2024-06-04T06:35:18]:INFO:Preparing Pathway computation\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "            | sum_profiles\n",
            "^X1MXHYY... | 101\n",
            "^YYY4HAB... | 1010\n"
          ]
        }
      ],
      "source": [
        "@pw.transformer\n",
        "class using_two_tables:\n",
        "    class profiles_table(pw.ClassArg):\n",
        "        profile: float = pw.input_attribute()\n",
        "\n",
        "    class matchings_table(pw.ClassArg):\n",
        "        a: pw.Pointer = pw.input_attribute()\n",
        "        b: pw.Pointer = pw.input_attribute()\n",
        "\n",
        "        @pw.output_attribute\n",
        "        def sum_profiles(self) -> float:\n",
        "            pa = self.transformer.profiles_table[self.a].profile\n",
        "            pb = self.transformer.profiles_table[self.b].profile\n",
        "            return pa + pb\n",
        "\n",
        "\n",
        "result = using_two_tables(\n",
        "    profiles_table=profiles, matchings_table=matchings\n",
        ").matchings_table\n",
        "pw.debug.compute_and_print(result)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "19",
      "metadata": {},
      "source": [
        "## Other topics\n",
        "\n",
        "We hope these examples make you feel comfortable using Pathway transformer classes. You can take a look at our advanced example of [transformer classes on a tree](/developers/user-guide/diving-deeper/transformer-recursion/).\n",
        "\n",
        "To continue your exploration of Pathway, you can also check out our [connectors](/developers/user-guide/connect/pathway-connectors/), or see directly how to use Pathway to implement classic algorithms such as [PageRank](/developers/showcases/pagerank)."
      ]
    }
  ],
  "metadata": {
    "jupytext": {
      "cell_metadata_filter": "-all",
      "main_language": "python",
      "notebook_metadata_filter": "-all"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.11.9"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}